The various possibilities for the progenitors of gamma-ray bursts (GRBs) manifest in differing observable properties. Through deep spectroscopic and high-resolution imaging observations of some GRB hosts, I demonstrate that well-localized long-duration GRBs are connected with otherwise normal star-forming galaxies at moderate redshifts of order unity. I test various progenitor scenarios by examining the offset distribution of GRBs about their apparent hosts, making extensive use of ground-based optical data from Keck and Palomar and space-based imaging from the Hubble Space Telescope. The offset distribution appears to be inconsistent with the coalescing neutron star binary hypothesis but statistically consistent with a population of progenitors that closely traces the ultra-violet light of galaxies. This is naturally explained by bursts which originate from the collapse of massive stars. This claim is further supported by the unambiguous detections of emission ''bumps'' which can be explained as supernovae that occur at approximately the same time as the associated GRB; if true, GRB 980326 and GRB 011121 provide strong observational evidence connecting cosmological GRBs to high-redshift supernovae and implicate massive stars as the progenitors of some long-duration GRBs. Interestingly, most alternative models of these bumps require wind-stratified circumburst media; this too, implicates massive stars. In addition to this work, I also constructed the Jacobs Camera (JCAM), a dual-beam optical camera for the Palomar 200-inch Telescope designed to follow-up rapid GRB localizations (abridged).